Unsupervised Kernel Dimension Reduction Supplemental Material

We start by noting that conditional independence X ⊥ Y |B⊤X does not necessarily imply the correlation between BX and Y is maximized. To see this, let X be a Gaussian random vairable with zero mean and diagonal covariance matrix. AssumeB is an identity matrix and Y = X = (BX) (elementwise square for a vectorial X). The conditional independence is obviously satisfied yet the correlation between BX and Y is zero. This observation is yet another example showing the limitation of Spearson’s correlation measures, which detect only linear dependence between random variables. In the following, we show that when measured in the RKHS, the two measures ĴY Y |X and ĴXY are equivalent. Assume we use Gaussian RBF kernel for both ĴY Y |X(B X,Y ) and ĴXY (B X,Y ): K(xi,xj) = exp ( −‖xi − xj‖/σ N )